Goto

Collaborating Authors

 pytorch code


Learning Bug Context for PyTorch-to-JAX Translation with LLMs

arXiv.org Artificial Intelligence

Despite recent progress of large language models (LLMs) on code translation among mainstream languages, translating PyTorch to JAX remains nontrivial. The two libraries, though both embedded in Python, differ in core design, execution semantics, and ecosystem maturity; JAX is newer and comparatively underrepresented in public code, and parallel PyTorch--JAX corpora are limited. Weaknesses in existing evaluation further complicate cross-framework benchmarking. We present T2J, a prompt-augmentation framework that strengthens LLM-based PyTorch to JAX translation. Our pipeline (i) assembles two PyTorch sources -- the problem-solving set from TorchLeet (Aroori & Chien, 2025) and a GitHub-derived set from CodeParrot (Wolf et al., 2022) -- and uses GPT-4o-mini to produce initial JAX drafts; (ii) engages two professional developers to iteratively repair those drafts until functional equivalence, yielding a curated fixed-bug dataset of common errors and patches; and (iii) constructs augmented prompts that inject structured guidance from these fixes to steer lightweight LLMs (e.g., GPT-4o-mini). We also introduce three metrics tailored to PyTorch to JAX: T2J CodeTrans Score, T2J FixCost Score (an LLM-based estimate of bug-fix effort), and T2J Comparison Score (LLM-as-judge). Empirically, T2J raises GPT-4o-mini performance by up to 10% on CodeBLEU, 50% on T2J FixCost Score, 1.33 points on T2J CodeTrans Score (0--4 scale), and 100% on T2J Comparison Score; moreover, the generated code runs up to 2.5x faster than the baseline.



Introduction to Lightning Fabric

#artificialintelligence

Lightning Fabric is a new, open-source library that allows you to quickly and easily scale models while maintaining full control over your training loop. In the past, getting PyTorch code to run efficiently on GPUs and scaling it up to many machines and large datasets was possible with PyTorch Lightning. As time went on, however, we became aware of the need to provide a scaling option that landed somewhere between a raw deep learning framework like PyTorch on the one hand, and a high-level, feature-rich framework like PyTorch Lightning. Lightning Fabric is just that. While PyTorch Lightning provides many features to save time and improve readability and collaboration, there are complex use cases where full control over the training loop is needed.


Does Zero-Shot Reinforcement Learning Exist?

arXiv.org Artificial Intelligence

A zero-shot RL agent is an agent that can solve any RL task in a given environment, instantly with no additional planning or learning, after an initial reward-free learning phase. This marks a shift from the reward-centric RL paradigm towards "controllable" agents that can follow arbitrary instructions in an environment. Current RL agents can solve families of related tasks at best, or require planning anew for each task. Strategies for approximate zero-shot RL ave been suggested using successor features (SFs) [BBQ+ 18] or forward-backward (FB) representations [TO21], but testing has been limited. After clarifying the relationships between these schemes, we introduce improved losses and new SF models, and test the viability of zero-shot RL schemes systematically on tasks from the Unsupervised RL benchmark [LYL+21]. To disentangle universal representation learning from exploration, we work in an offline setting and repeat the tests on several existing replay buffers. SFs appear to suffer from the choice of the elementary state features. SFs with Laplacian eigenfunctions do well, while SFs based on auto-encoders, inverse curiosity, transition models, low-rank transition matrix, contrastive learning, or diversity (APS), perform unconsistently. In contrast, FB representations jointly learn the elementary and successor features from a single, principled criterion. They perform best and consistently across the board, reaching 85% of supervised RL performance with a good replay buffer, in a zero-shot manner.


GitHub - rentruewang/koila: Prevent PyTorch's `CUDA error: out of memory` in just 1 line of code.

#artificialintelligence

Koila solves CUDA error: out of memory error painlessly. Fix it with just one line of code, and forget it. To use it, download release v0.1.1 here.* Automatically accumulates gradients when batch sizes are too large. Lazily evaluates PyTorch code to save computing power.


Learn PyTorch for Deep Learning โ€“ Free 26-Hour Course

#artificialintelligence

My comprehensive PyTorch course is now live on the freeCodeCamp.org The best way to learn is by doing. And that's just what we'll do in the Learn PyTorch for Deep Learning: Zero to Mastery course. If you're new to data science and machine learning, consider the course a momentum builder. By the end, you'll be comfortable navigating the PyTorch documentation, reading PyTorch code, writing PyTorch code, searching for things you don't understand and building your own machine learning projects.


Learn PyTorch in a day. Literally.

#artificialintelligence

My latest YouTube video is 25 hours, 36 minutes and 57 seconds long (actually a little over a day). And its sole purpose is to be a momentum builder, to help you learn PyTorch for deep learning. In each section, we'll get hands-on and learn important machine learning concepts by writing PyTorch code together, apprenticeship style. It still amazes me to think how much the field of machine learning is progressing. There are things in the video that weren't possible 10 years ago.


Scale your PyTorch code with LightningLite

#artificialintelligence

LightningLite comes with distributed utilities and basic built-in properties that you can rely on to ensure your distributed training works as expected. LightningLite provides you only with the tool to scale your training. You need to know several significant challenges ahead of you, but they are already resolved in PyTorch Lightning. If you are facing one of those challenges, then you are already meeting the limit of LightningLite. We recommend you convert to Lightning, so you never have to worry about those.


Best Machine Learning Research of 2020

#artificialintelligence

We saw excellent progress with enterprise acceptance of machine learning across a wide swath of industries and problem domains. In terms of pure research, I had a good time tracking the acceleration of progress in the area of machine learning. In this article, we'll take a tour of my top pick of papers that I found intriguing and useful. In my attempt to stay current with the field's research progress, the directions represented here are very promising. I hope you enjoy the results as much as I have. Overfitting & underfitting and stable training are important challenges in machine learning. Current approaches for these issues are mixup, SamplePairing, and BC learning. This paper states the hypothesis that mixing many images together can be more effective than just two.


A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

arXiv.org Artificial Intelligence

Black-box machine learning learning methods are now routinely used in high-risk settings, like medical diagnostics, which demand uncertainty quantification to avoid consequential model failures. Distribution-free uncertainty quantification (distribution-free UQ) is a user-friendly paradigm for creating statistically rigorous confidence intervals/sets for such predictions. Critically, the intervals/sets are valid without distributional assumptions or model assumptions, with explicit guarantees with finitely many datapoints. Moreover, they adapt to the difficulty of the input; when the input example is difficult, the uncertainty intervals/sets are large, signaling that the model might be wrong. Without much work, one can use distribution-free methods on any underlying algorithm, such as a neural network, to produce confidence sets guaranteed to contain the ground truth with a user-specified probability, such as 90%. Indeed, the methods are easy-to-understand and general, applying to many modern prediction problems arising in the fields of computer vision, natural language processing, deep reinforcement learning, and so on. This hands-on introduction is aimed at a reader interested in the practical implementation of distribution-free UQ, including conformal prediction and related methods, who is not necessarily a statistician. We will include many explanatory illustrations, examples, and code samples in Python, with PyTorch syntax. The goal is to provide the reader a working understanding of distribution-free UQ, allowing them to put confidence intervals on their algorithms, with one self-contained document.